01. From RL to Deep RL

From RL to Deep RL

So far, you've solved many of your own reinforcement learning problems, using solution methods that represent the action values in a small table. Earlier in the nanodegree, we referred to this table as a Q-table.

In the video below, Kelvin Lwin will introduce you to the idea of using neural networks to expand the size of the problems that we can solve with reinforcement learning. This context is useful preparation for exploring the details behind the Deep Q-Learning algorithm later in this lesson!

Kelvin is a Senior Deep Learning Instructor at the NVIDIA Deep Learning Institute.

From RL to Deep RL

## Stabilizing Deep Reinforcement Learning

As you'll learn in this lesson, the Deep Q-Learning algorithm represents the optimal action-value function q_* as a neural network (instead of a table).

Unfortunately, reinforcement learning is notoriously unstable when neural networks are used to represent the action values. In this lesson, you'll learn all about the Deep Q-Learning algorithm, which addressed these instabilities by using two key features:

  • Experience Replay
  • Fixed Q-Targets

Watch the video below to learn more!

DQN Overview

## Additional References